An Ensemble Classification Algorithm Based on Information Entropy for Data Streams

Data stream mining problem has caused widely concerns in the area of machine learning and data mining. In some recent studies, ensemble classification has been widely used in concept drift detection, however, most of them regard classification accuracy as a criterion for judging whether concept drift happening or not. Information entropy is an important and effective method for measuring uncertainty. Based on the information entropy theory, a new algorithm using information entropy to evaluate a classification result is developed. It uses ensemble classification techniques, and the weight of each classifier is decided through the entropy of the result produced by an ensemble classifiers system. When the concept in data streams changing, the classifiers’ weight below a threshold value will be abandoned to adapt to a new concept in one time. In the experimental analysis section, six databases and four proposed algorithms are executed. The results show that the proposed method can not only handle concept drift effectively, but also have a better classification accuracy and time performance than the contrastive algorithms. J. Wang 1. School of Computer and Information Technology, Shanxi University, Taiyuan, China 2. Key Laboratory of Computational Intelligence and Chinese Information Processing, Ministry of Education, Taiyuan, China Tel.: +86 0351-7010566 E-mail: [email protected] S. Xu School of Computer and Information Technology, Shanxi University, Taiyuan, China E-mail: [email protected] B. Duan School of Computer and Information Technology, Shanxi University, Taiyuan, China E-mail: [email protected] C. Liu School of Computer Science and Technology, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, China E-mail: [email protected] J. Liang Key Laboratory of Computational Intelligence and Chinese Information Processing, Ministry of Education, Taiyuan, China E-mail: [email protected] ar X iv :1 70 8. 03 49 6v 1 [ cs .D S] 1 1 A ug 2 01 7 2 Junhong Wang et al.